Field of the Invention
Embodiments described herein relate to color balancing techniques, and more specifically to color balancing an image (or frame) based on comparisons between the image and a reference image.
Description of the Related Art
Computer graphics technology has come a long way since video games were first developed. Relatively inexpensive 3D graphics engines now provide nearly photo-realistic interactive game play on hand-held video game, home video game and personal computer hardware platforms costing only a few hundred dollars. These video game systems typically include a hand-held controller, game controller, or, in the case of a hand-held video game platform, an integrated controller. A user or player uses the controller to send commands or other instructions to the video game system to control a video game or other simulation being played. For example, the controller may be provided with a manipulator (e.g., a joystick) and buttons operated by the user.
Many hand-held gaming devices include some form of camera device which may be used to capture an image or a series of images of a physical, real-world scene. The captured images can then be displayed, for instance, on a display of the hand-held gaming device. Certain devices may be configured to insert virtual objects into the captured images before the images are displayed. Additionally, other devices or applications may enable users to draw or paint particular within a captured image of a physical scene. However, as such alterations apply only to a single image of the physical scene, subsequent captured images of the physical scene from different perspectives may not incorporate the user's alterations. Furthermore, alterations by the system (e.g., a camera device that is currently filming a scene) may automatically adjust visual factors such as white balance, brightness, contrast and other various image parameters, in order to optimize the camera device under varying lighting conditions. However, because such alterations are performed at the camera level, they do not apply to virtual objects inserted into the visual scene. Moreover, such alterations may differ from one camera device to another, depending on the lighting conditions of the specific camera. As such, when compositing two or more video streams into a single video stream, these varying image parameters can cause distortions and other problems in the composited video stream.
Digital images are used in a wide array of different applications. In many applications, some form of image analysis and manipulation can be performed to enhance the picture quality of the digital images. For example, the quality of a digital image may be dependent on color levels and color intensity within the image, as unrealistic coloration within an image may degrade the visual quality of the image. When capturing a digital photo (e,g., with a digital camera device or a mobile device configured with a digital camera), characteristics of the ambient environment, e.g., lighting, reflections, etc., may impact the coloration of the digital picture, and in some cases may result in an unrealistic coloration for certain objects in the digital picture.
To address this, the coloration of a photographically recorded digital image can be adjusted using various color balancing techniques. Generally, color balancing involves altering the coloration of at least part of an image (or a frame in a video) in a deliberate fashion. However, while many imperfections may be corrected through the use of color balancing techniques, which aspects of the image, e.g., hue, saturation, chrome, colorfulness, value or lightness, luminance, etc., need to be adjusted to create realistic coloration can vary dramatically.
Unfortunately, many conventional color balancing techniques may provide inadequate results. For instance, because the coloration adjustments that need to be made vary so much from one image to another, many color balancing techniques rely on some form of manual user input or intervention in order to achieve a suitable color balance. As another example, other color balancing techniques may apply a predetermined color balancing operation to an image, and may then display the altered image to a user. In such a system, the user can then manually change the coloration of the image as is necessary to produce a more realistic image (e.g., an image whose coloration more realistically reflects the coloration of the original scene captured by the image). While these techniques may produce images with realistic coloration, they are ill-suited for realtime applications, such as color balancing frames within a video stream being displayed in realtime. In addition to the real-time aspect, these manual interfaces are generally less intuitive to use. That is, with conventional techniques, abstract tools such as changing saturation, hue, contrast, gamma and more general “tone curves” are used for color balancing purposes. However, none of these tools allow a user to map colors within one image onto the colors of another image (e.g., a particular red coloration in image A should look like a slightly different red coloration in image B, a green coloration in image A should actually look like a blue coloration in image B, etc.)